This invention is directed to the field of computer networks. It is more particular directed to computer servers, server clusters, server content and server administration.
There is a continuous desire to reduce the time it takes for a client to obtain the content it requests from a server. This applies for example both within the context of a corporate Intranet as well as the Internet. An example of a typical server is a web server used within the context of a web site. A common problem among high performance and/or popular web sites is that the web servers cannot serve pages at a fast enough rate when the maximum capacity of a particular web server is reached. A common architecture for high performance web sites consists of a set of web servers using a static configuration for web content. Other sites often use one of the methods described below.
One method provides load balancing by having a network dispatcher first receive web based client packets on an ingress interface. The dispatcher modifies the destination address to be that of one particular web server from the set of web servers located at the egress point of the network dispatcher. The network dispatcher determines which web server is least heavily loaded in order to forward the client""s request to that server. The network dispatcher is generally limited to a local area network (LAN) environment in that all web servers are at a point behind the network dispatcher. In addition, the network dispatcher becomes the bottleneck for all network traffic destined to the servers and needs to have a minimum connection rate which is the sum of the connection rates of all the servers it is handling for optimal performance. Also, the dispatcher becomes a single point of failure. Given these issues, use of the network dispatcher method does not scale well.
Still another approach uses a dynamic Domain Name Service (DNS). This approach is accomplished by having a dynamic DNS server which receives requests from clients requesting a network address of well known host names. The DNS server responds to the client by selecting one network address from a set of available addresses for the particular host name typically in a round robin manner. This approach allows a server""s name to be mapped to a set of multiple servers, each with a different network address. This approach is limited in that it does not take into account the current or recent load on the server.
Another approach uses a broadcast medium whereby the client""s request is broadcast to all servers within a group. Each server decodes the client""s address and makes a determination as to whether or not to handle this request based on the client""s address. This implementation requires that all web servers listen to all client requests and discard those that are not within the configured range.
It is therefore an aspect of the present invention to provide a method for restructuring server content so as to distribute network traffic over a group of servers. The method provides a monitoring process which collects log files from a set of servers and analyzes them to find file relations. It merges files into groups based on a relation and on a specified configuration of the servers. The analysis also includes employing measurements of server traffic on a file basis and establishing a traffic load target for each server. The results of the analysis are then distributed to the set of servers within the group where the content is either distributed to a specific server or made active on a specific server.
It is another aspect of the invention to provide a method whereby files are relocated among servers and links to file locations are updated to show the new location of the files.
It is a further aspect of the invention to provide an apparatus, a computer product and an article of manufacture comprising a computer usable medium having computer readable program code means embodied therein for causing a computer process to be capable of restructuring and distributing the content among a set of servers. These implement a method such as to enable dynamic load balancing of a set of servers in a server cluster.
In an example of the invention, a web site monitoring process collects web server logs from a set of web servers. An algorithm is run on the web server logs to determine the number of times each file is requested and the size of each request and to determine any related files. The monitoring process creates groups of files based on these criteria and further combines groups of files until there exists a set of N groups of files where N is the number of servers in the cluster. All of the newly created sets of files are communicated to all servers in the group through an auxiliary process, or though the CGI based URL in the case of a web server. The content at each server is made active by transferring the content to each server.
In an alternative embodiment all content is stored on every server. Only a subset of the stored files is activated on each server as required by the utilization. Each server searches the content local to itself and modifies any content having a link to files that have been moved or deactivated by the monitoring process to another server in the group.
In an example embodiment the server site monitor includes a server log store to store log file information about each server in a server cluster, a file statistic database used to build relations about files, and a file group database where files groups are stored in order to send group information to each server within the server cluster.